Image processing device

ABSTRACT

This is an image processing device for predicting the lower-order bits of target pixel data, based on one or more pieces of pixel data constituting image data. The image processing device comprises a lower-order bit calculation unit for calculating the lower-order bits of the target pixel data, based on one or more pieces of pixel data constituting image data and specifying the data as corrected lower-order bits and a lower-order bit superimposition unit for superimposing the corrected lower-order bits calculated by the lower-order bit calculation unit on higher-order bits of the target pixel data.

CROSS REFERENCE TO RELATED APPLICATION

This application claims benefit of Japanese Application No. 2004-200735,filed Jul. 7, 2004, the contents of which are incorporated by thisreference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing device, such as adigital camera or the like, and more particularly, it relates to animage processing technology capable of improving S/N aftercompressing/expanding conversion and obtaining a favorable gradationcharacteristic.

2. Description of the Related Art

Recently, with the advance of image input equipment, such as a digitalcamera or the like, it has become possible to take in image data withhigh accuracy. Such image data taken in with high accuracy can be storedin a storage device, and a user can easily output the image data to anoutput device, such as a monitor, a printer or the like, by a personalcomputer or the like.

When storing image data, the image data is sometimes stored as digitalsignals of 12-14 bit per pixel. For example, if it is 12 bits, thenumber of gradation is 4096. In this way, the image data of 12-14 bitstaken in with high accuracy is stored in a storage medium after it isnot compressed or reversibly compressed.

In this case, if the image data of 12 bits per pixel is stored withoutbeing processed, the amount of the data becomes enormous. Therefore, theimage data is stored in the storage medium after it is compressed.

As a method for storing images shot by an image input device, such as adigital camera or the like, there is a method for anticipating that thelower-order bits of image data are camera noise or the like and cuttingthem away. As the method for cutting lower-order bits, a method forcutting away the entire pixel and a method for modifying lower-orderbits to be cut away case by case taking into considering the amount ofnoise are proposed.

Japanese Patent Application No. 2000-244922 discloses a method forcutting away lower-order bits, using the number of bits corresponding toa valid data value and setting the number of bits for each pixel (methodfor correlating lower-order bits to higher-order bits).

However, if image processing is applied to compressed data obtained bycutting away lower-order bits after it is being expanded, the imageprocessing is applied with the reduced number of gradation due to thecutting of bits. Therefore, when compared with a case with no cutting ofbits, an unnatural change of level (tone jump) occurs in the edge andgradation changing part of an image.

SUMMARY OF THE INVENTION

One aspect of the present invention is an image processing device forpredicting the lower-order bits of target pixel data, based on one ormore pieces of pixel data constituting the image data. The imageprocessing device comprises a lower-order bit calculation unit forcalculating the lower-order bits of the target pixel data, based on theone or more of pieces of pixel data constituting the image data andspecifying them as corrected lower-order bits and a lower-order bitsuperimposition unit for superimposing the corrected lower-order bitscalculated by the lower-order bit calculation unit on higher-order bitsof the target pixel data.

It is preferable for the lower-order bit calculation unit to calculatethe corrected lower-order bits, based on the higher-order bits of thepixel data.

It is preferable for the lower-order bit calculation unit to determine aweighting coefficient of the image data and to calculate the correctedlower-order bits by multiplying the higher-order bits of the targetpixel data by the weighting coefficient.

It is preferable for the lower-order bit calculation unit to calculatecorrelation between pixel values using pixel data in the neighborhood ofthe target pixel data and to calculate the corrected lower-order bits,based on the correlation.

It is preferable for the lower-order bit calculation unit to calculatethe correlation by averaging higher-order bits of adjacent pixels oneach side of the target pixel.

It is preferable for the lower-order bit calculation unit to calculatethe correlation by averaging higher-order bits of pixels in theneighborhood of the target pixel.

It is preferable for the lower-order bit calculation unit to calculate afirst color component correlation, based on the higher-order bits of apixel with a different color component in the primary-color Bayer arrayin the neighborhood of the target pixel, to further calculate a secondcolor component correlation, based on the result of the first colorcomponent correlation result and the higher-order bits of the pixel andto calculate the corrected lower-order bits, based on the second colorcomponent correlation.

It is preferable for the lower-order bit calculation unit to calculatingthe first color component correlation by calculating a function based ona correlation using the higher-order bits of pixels of the entire imagedata.

It is preferable for the lower-order bit calculation unit to calculatethe first color component correlation by dividing the image data andcalculating a function, based on the correlation using the higher-orderbits of the pixel for each divided block.

It is preferable for the lower-order bit calculation unit to calculatethe second color component correlation by calculating the correctedlower-order bits, using a table prepared beforehand using the valuescalculated based on the first color component correlation.

Another aspect of the present invention is a computer readable storagemedium storing a program for enabling a computer to execute an imageprocessing. The image processing comprises a lower-order bit calculationstep of predicting the lower-order bits of target pixel data, based onone or more pieces of pixel data constituting image data, calculatingthe lower-order bits of the target pixel data, based on the one or morepieces of pixel data constituting the image data and specifying them ascorrected lower-order bits, and a lower-order bit superimposition stepof superimposing the corrected lower-order bits calculated by thelower-order bit calculation step on higher-order bits of the targetpixel.

It is preferable to calculate the corrected lower-order bits, based onhigher-order bits of the pixel data.

Another aspect of the present invention is a computer data signalembodied in a carrier wave, for representing a program for enabling acomputer to execute an image processing. The image processing comprisesa lower-order bit calculation step of predicting the lower-order bits oftarget pixel data, based on one or more pieces of pixel dataconstituting image data, calculating the lower-order bits of the targetpixel data, based on the one or more pieces of pixel data constitutingthe image data and specifying them as corrected lower-order bits, and alower-order bit superimposition step of superimposing the correctedlower-order bits calculated by the lower-order bit calculation step onhigher-order bits of the target pixel data.

It is preferable to calculate the corrected lower-order bits, based onhigher-order bits of the pixel data.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows the configuration of the digital camera for implementingthe present invention.

FIG. 2 is a flowchart showing the image processing.

FIG. 3 is a flowchart showing the image processing (variation).

FIG. 4 shows the compression process of pixel data.

FIG. 5 shows the expansion process of pixel data.

Fig. 6 shows a method using correlation with higher-order bits.

FIG. 7 shows the configuration of the method using correlation withhigher-order bits.

FIG. 8 is a flowchart showing the method using correlation withhigher-order bits.

FIG. 9 shows the array of adjacent pixel data on each side.

FIG. 10 shows a method using correlation with adjacent pixel data oneach side.

FIG. 11 shows the configuration of the method using correlation withadjacent pixel data on each side.

FIG. 12 is the flowchart of the method using correlation with adjacentpixel data on each side.

FIG. 13 shows the array of adjacent pixel data on each side.

FIG. 14 shows a method using correlation with adjacent pixel data to theleft, on the above and to the upper right.

FIG. 15 shows the configuration of the method using correlation withadjacent pixel data to the left, on the above and to the upper right.

FIG. 16 is the flowchart of the method using correlation with adjacentpixel data to the left, on the above and to the upper right.

FIG. 17 shows the primary-color Bayer array.

FIG. 18 shows a method using correlation with color components in theneighborhood.

FIG. 19 shows the configuration of the method using correlation withcolor components in the neighborhood.

FIG. 20 is the flowchart of the method using correlation with colorcomponents in the neighborhood.

FIG. 21 shows examples of the computer-readable storage medium on whichis recorded a control program.

DESCRIPTION OF THE PREFERRED EMBODIMENTS The First Preferred Embodiment

The preferred embodiments of the present invention are described belowwith reference to the drawings. FIG. 1 shows the configuration of thedigital camera for implementing the present invention. Although in thispreferred embodiment, a digital camera is used as an image processingdevice, it is not limited to a digital camera.

A central processing unit (CPU)-A 1 controls the operation of thisentire digital camera. Electrically erasable and programmable read-onlymemory (EEPROM) 2 stores a control program for enabling the CPU-A 1 tocontrol the operation of the digital camera. The CPU-A 1 performs theabove-mentioned control by executing this control program.

A lens 3 is an image sensing optical system for forming the image of anobject on a charge coupled device (CCD) 4. The CCD 4 is an image sensingdevice for photo-electrically converting the formed image of an objectand outputting electrical signals (analog signals) representing animage. An image sensing circuit 5 comprises a correlated double sampling(CDS) circuit, an automatic gain control (AGC) circuit and the like. Theimage sensing circuit 5 eliminates reset noise contained in theelectrical signal outputted from the CCD 4, adjusts their signal levelsand so on, under the control of the CPU-A 1.

An A/D conversion unit 6 converts the output signals (analog signals) ofthe image sensing circuit 5 into digital data. A correction circuit 7applies a correction process, such as white balance, Y correction andthe like, to a camera image represented by the digital data outputtedfrom the A/D conversion unit 6.

The output of the correction circuit 7 is connected to frame memory 8, aCPU-B 9, first-in first-out (FIFO) memory 10 and a storage mediuminterface (I/F) 11 via a bus 12, and data can be transmitted/receivedbetween them under the control of the bus 12 by the CPU-B 9.

The frame memory 8 is used as buffer memory for temporarily storingcamera images outputted from the correction circuit 7 and ones to whichthe CPU-B 9 has applied an image processing. Simultaneously, it is alsoused as a working memory area for a variety of processes by the CPU-B 9.

The CPU-B 9 performs the management of the frame memory 8 according tothe control program provided inside (frame memory control function),applies the expansion process to compressed image data (image expansionfunction), applies a data compression process to a camera image (imagecompression function), performs an access control process to a storagemedium 13 in order to write and read data (storage medium accessfunction) and the like, under the control of the CPU-A 1.

In the compression and expansion processes of image data, for example,compression and expansion processes are performed by joint photographicexperts group (JPEG) method.

The FIFO memory 10 is used to temporarily store image data representingan image to be displayed, and stores image data after the CPU-B hasapplied an image processing to it. The storage medium I/F 11 provides aninterface function to transmit/receive data to/from the storage medium13, and writes/reads data into/from the storage medium 13.

The storage medium 13 is used to store recorded data. A removablestorage medium, such as a memory card in which semiconductor memory isbuilt, or the like, is convenient for a digital camera. A video encodingcircuit 14 superimposes on-screen information, such as date or the like,on an image represented by data read from the FIFO memory 10 under thecontrol of the CPU-A 1.

A video output circuit 14 converts digital data outputted from the videoencoding circuit 14 into video signals (image signals), being analogsignals. A video output terminal 16 outputs video signals outputted fromthe video output circuit 15 to the outside of this digital camera.

A thin film transistor (TFT) liquid crystal device (LCD) driving circuit17 drives a TFT panel 18, based on image signals outputted from thevideo encoding circuit 14. The TFT panel (TFT LCD panel) 18 is an LCDmonitor driven by the TFT LCD driving circuit 17, for displaying animage represented by image signals outputted from the video encodingcircuit 14.

This digital camera is provided with an optical finder, which is notshown in FIG. 1, and the object image formed on the CCD 4 can be shotwithout displaying it on an LCD monitor. Therefore, a user can setwhether the LCD monitor should be operated, in this digital camera. Inthis case, if the user sets in such a way that the LCD monitor shouldnot be operated, power consumption can be reduced at the time of thenormal operation of this digital camera.

A backlight unit 19 constitutes the LCD monitor together with the TFTpanel 18, and is provided with a light source for illuminating the LCDpart of the TFT panel 18 from the back under the control of the CPU-A 1.As this light source, a white light emitting diode (LED) or the like isused.

An actuator 20 is an operating device, such as a motor for driving thelens (camera optical system) 3 (an AF motor, a zooming motor, etc), amechanical shutter driving device, and is driven by an actuator drivingcircuit 21. the actuator driving circuit 21 drives the actuator 20 underthe control of the CPU-A 1.

An external data I/F 22 provides an interface function totransmit/receive data between an external device, such as a personalcomputer or the like, and the CPU-A 1, and is, for example, a universalserial bus (UBS) type I/F. A key matrix unit 23 can be obtained bywiring a variety of keys, switches, buttons and the like for receiving avariety of instructions from a user and notifying the CPU-A 1 of them.This key matrix unit 23 also includes a release button which is operatedto enable this digital camera to shoot an image. The CPU-A 1 can detectthe first release (so called half shutter) and second release (so calledfull shutter) of the release button.

An LCD display circuit 24 drives an LCD panel 25 under the control ofthe CPU-A 1. The LCD panel 25 is driven by the LCD display circuit 24,and displays a variety of setting statuses of this digital camera, suchas exposure, shutter speed or the like, according to its driving signal,and the number of images which can be currently shot or recorded.

A battery 26 supplies this digital camera with power. A power supplycircuit 27 controls voltage supplied from the battery 26 to supply eachunit of the digital camera with power. The CPU-A 1 monitors the voltageof the battery 26 for supplying the power supply circuit 27 with power.If its voltage value even temporarily drops below a predetermined valuewhen this digital camera normally operates (at the time of power-on),the CPU-A 1 prohibits this digital camera to shoot after that and doesnot perform a shooting process even if the release button is operated.

In this case, a general personal computer can also be used for the CPU-A1, frame memory 8, CPU-B 9, TFT LCD driving circuit 17 and TFT panel 18.

Processes described later can be performed by a computer program, andthe process program is recorded on a storage medium, such as CD-ROM,etc., a hard disk or the like. The process program on the memory of thepersonal computer is read as requested, and the CPU executes it tocontrol each device connected to the personal computer.

Alternatively, the storage medium can be a storage device which isprovided for a computer, is connected to the computer via acommunication line and functions as a program server.

In this case, a transmission signal which can be obtained by modulatinga carrier wave by a data signal representing the control program istransmitted from the program server to the computer via thecommunication line, being a transmission medium. In the computer, theCPU can execute this process program by demodulating the receivedtransmission signal and reproducing the process program.

Preferred Embodiment 1

Next, the image processing of the present invention in the digitalcamera with such a configuration is described. In this preferredembodiment, 12-bit pixel data composed of higher-order bits a (10 bits)and lower-order bits b (2 bits) are processed by a method usingcorrelation with higher-order bits.

FIGS. 2 and 3 are flowcharts of the image processing. FIGS. 2 and 3 showthat calculated lower-order bits (corrected lower-order bits) are addedto (superimposed on) an image stored as pixel data obtained by cuttingaway the lower-order bits of the data bits of one pixel as camera noiseas superimposition data instead of the cut bits (lower-order bits).

The flowchart shown in FIG. 2 is described below.

In step S21, the superimposition process of lower-order bits is started,and pixel data whose lower-order bits are cut away and compressed priorto S21 is handed.

In step S22, the compressed data is expanded, and in step S23, theentire bits are shifted to the left side by the cut lower-order bits.

Then, in step S24, data to be superimposed as lower-order bits iscalculated, and in step S25, the superimposition data is added.

The processes in steps S23 through S25 are applied to all pixels. If instep S26 it is determined that all the pixels are process, the processproceeds to YES and terminates. If it is NO, the processes in steps S23through S25 are repeated.

FIG. 3 shows a variation of FIG. 2, in which the order of processes isreplaced. The flowchart shown in FIG. 3 is described below. In stepsS21, S22 and S26, the same processes as in FIG. 2 are performed, and theorder of the processes in steps S23 through S25 is replaced.

In FIG. 3, the process in step S24 is firstly performed and data to besuperimposed as lower-order bits is calculated. Then, in step S23 theentire bits are shifted to the left side by the cut lower-order bits.Then, in step S25, the superimposition data is added. Therefore, theresult of FIG. 3 is the same as that of FIG. 2.

Next, the above-mentioned process is described in more detail withreference to FIGS. 4 and 5. In FIG. 4, the 12-bit pixel data composed ofthe higher-order bits a (10 bits) and lower-order bits b (2 bits) aretaken into the CPU-B 9, and the lower-order bits b are cut way by theimage compression function to be modified to 100 bit data. Then, acompression process is applied, and the pixel data is stored. So far theprocess in step 21 is described.

Then, in FIG. 5, when outputting the image data, such as when displayingthe image data on the TFT panel 8 or so on, the data compressed andstored in step S21 is called up, is expanded by the expansion process ofthe CPU-B 9 to be converted into 10-bit data. Then, the higher-orderbits a are shifted by two bits, and the superimposition data calculatedin step S24 is superimposed as lower-order bits to generate 12-bit data.

Next, the superimposition data calculated in S24 is described. In thispreferred embodiment 1, a method for superimposing superimposition datarandomly calculated as lower-order bits is used, which is described withreference to FIG. 6.

In S61, the expanded 10-bit pixel data two bits of which are cut awaywhen compressed is shifted to the left side by two bits.

Then, in S62, 0 is superimposed in the places of two lower-order bits b0and b1. In this case, 0 is superimposed in each of b0 and b1.

In S63, lower-order bits b are calculated according to equation (1) ofS63. Firstly, equation (1) of S63 calculates ω(a) by comparing them withthe higher-order bits a. If the higher-order bits a are 0 or 1, equation(2) of S63 sets the lower-order bits cut away when compressed to 0.

If the higher-order bits a are other than 0, equation (3) randomlyselects superimposition data. In this case, if the lower-order bits bare two bits, ω(a) selects values 00, 01, 10 and 11, and calculatesequation (1) of S63, b=ω(a)·a to specify the lower-order bits of themultiplied 12-bit data as b. If they are two bits, its higher-order bitscan also be selected as b.

In S64, the lower-order bits b calculated in S63 are superimposed on thehigher-order bits a as superimposition data.

FIG. 7 shows the configuration for executing S61 through S64 shown inFIG. 6. The cut 10-bit pixel data is expanded by an expansion unit 71.Then, a bit shift unit 72 performs the processes in S61 and S62, anddata memory 73 stores the result of S62.

Then, a data comparison unit 74 calculates ω(a), based on thehigher-order bits a according to the conditions of S63. Then, asuperimposition data calculation unit 75 calculates equation (1) of S63to calculate the lower-order bits b. After the calculation, theoperation shown in S64 are performed. Specifically, the superimpositiondata is superimposed on (added to) the left-shifted 10-bit pixel datastored in the data memory to generate 12-bit pixel data.

Next, the above-mention configuration is described with reference to theflowchart shown in FIG. 8.

In step S81, it is determined whether lower-order bits are cut away whencompressing pixel data. If they are already cut away, the processproceeds to S82. If they are not cut away yet, the process proceeds toS83. In this case, information for determining whether lower-order bitsare cut away is stored as part of compression information, such as thenumber of bits cut away when compressed, and the determination is madeby referring to this information. A user can select how to handle thelower-order bits cut away when compressed. Alternatively, conditions canbe determined based on the image state of all images, and it can beautomatically selected based on the conditions.

In step S82, the number of bits to be cut away determined whencompressed is read, and in step S83, since they are not cut away, thenumber of bits to be cut away is set to 0.

In step S84, all pixels contained in one image whose lower-order bits bare cut away and compressed are expanded to be converted into the cutpixel data before compression (10 bits).

In step S85 the data expanded in S84 is read, and in step S86 it isdetermined whether the cut bits are 0. If it is YES, the processproceeds to S811 without the superimposition process of lower-orderbits. If it is NO, the process proceeds to step S87, and the entire bitsare shifted to the left side by the cut bits. Then, in step S88 it isdetermined which the higher-order bits a of the data left-shifted in S87is, 0, 1 or other than 0 and 1. if they are 0 or 1, it is YES and theprocess proceeds to S811. If it is NO, the process proceeds to S89, and00, 01, 10 and 11 are selected as the value of ω(a), according to theconditions of S63. Then, equation (1) of S63 is calculated to calculatethe lower-order bits b. Then, in step S810, the lower-order bits b aresuperimposed on the higher-order bits a by adding them.

In step S811, it is determined whether the process of all necessarypieces of pixel data is completed. If it is completed, the processproceeds to YES and terminates. If is not completed, the processproceeds to NO, the processes in S85 and after are repeated.

Although in this preferred embodiment 1, the lower-order bits b arecalculated as ω(a)·a, the value of ω(a) can also be directlysuperimposed as lower-order bits.

Although in this preferred embodiment, the bit width of pixel data is 12bits and is composed of 10 higher-order bits and 2 lower-order bits, thecomposition is not limited to this. It can be modified as long as theedge of an image and the discontinuity of gradation can be eliminated,further S/N can be improved and a favorable gradation characteristic canbe obtained.

For example, weight can be randomly placed in the neighborhood offrequently appearing value in image data and 0 can be placed in theother part.

The edge of an image and the discontinuity of gradation can beeliminated and S/N can be improved by adding target image data using arandom value and a fixed value as weighting coefficients, generatinglower-order bits with the same bit width as those cut away and addingthem.

Preferred Embodiment 2

In this preferred embodiment, 12-bit pixel data composed of higher-orderbits a_(n) (10 bits) and lower-order bits c_(n) (2 bits) are processedby a method using the correlation with adjacent color components.

FIG. 9 shows adjacent pixels in Bayer array. In order to determine thelower-order bits c_(n) of a pixel an using the correlation with adjacentcolor components, the lower-order bits c_(n) is calculated anddetermined based on the correlation in pixel data between thehigher-order bits a_(n−1) of a pixel adjacent to the pixel on the leftside and the higher-order bits a_(n+1) of a pixel adjacent to the pixelon the right side.

A method for calculating the lower-order bits c_(n), based on thecorrelation with adjacent color components is described below withreference to FIG. 10.

In S101, the expanded 10-bit pixel data two bits of which are cut awaywhen compressed is shifted to the left side by two bits. The entire bitis shifted to the MSB side by two bits by shifting the bits in S101.Then, in S102, two lower-order bits b0 and b1 are set to 0. In thiscase, each of b0 and b1 is set to 0.

In S103, b_(n) is calculated according to equation (1) of S103. Firstly,b_(n)=(a_(n−1)+a_(n+1))/2 is calculated using the higher-order bitsa_(n−1) and a_(n+1) (10-bit pixel data) of adjacent pixel data. Then, avalue meeting the conditions is determined based on the result of b_(n)by referring to equations (2) through (5) of S103. Equation (2) of S103compares b_(n) with a preset threshold value 1 (k₁). If b_(n)>k_(i),c_(n) is set to 3 ([b0][b1]→[1][1]).

Equation (3) of S103 compares b_(n) with preset threshold values 1 (k₁)and 2 (k₂). If k₂<b_(n)<k₁, c_(n) is set to 2([b0] [b1]→[1]→[0]).

Equation (4) of S103 compares b_(n) with preset threshold values 2 (k₂)and 3 (k₃). If k₃<b_(n)<k₂, c_(n) is set to 2([b0][b1]→[0][1]).

Equation (5) of S103 compares b_(n) with a preset threshold value 3(k₃). If b_(n)<k₃, c_(n) is set to 0([b0][b1]→[0][0]).

Then, in S104, the corrected lower-order bits c_(n), beingsuperimposition data determined in S103, is superimposed as thelower-order bits c_(n) of the pixel data.

In this case, each of the preset threshold values k₁, k₂ and k₃ is setin the range between the maximum value (all higher-order bits are 1) andthe minimum value (all higher-order bits are 0) of the averages of thehigher-order bits of adjacent pixel data on each side. A user can alsoset this is such a way to obtain a desirable image.

FIG. 11 shows the configuration for executing S101 through S104 shown inFIG. 10. An expansion unit 111 expands the 10-bit pixel data the twobits of which are cut away when compressed. Then, a bit shift unit 112performs the processes in S101 and S102. Data memory 113 stores theresult of S102. An average calculation unit 114 calls up a_(n−1) anda_(n+1) stored in an adjacent pixel data memory 115, and calculatesb_(n) according to equation (1) of S103. Then, a superimposition datacalculation unit 116 determines c_(n) according to the conditions ofequations (2) through (5) of S103. After the calculation, 12-bit pixeldata can be generated by adding the superimposition data and the 10-bitleft-shifted pixel data an stored in the data memory by the operationshown in S104.

Next, the flowchart shown in FIG. 12 is described below.

In step S121, it is determined whether lower-order bits are cut awaywhen compressing pixel data. In this case, information for determiningwhether lower-order bits are cut away is stored as part of compressioninformation, such as the number of bits cut away when compressed, andthe determination is made by referring to this information. If they arealready cut away, the process proceeds to S112. If they are not cut awayyet, the process proceeds to S123. In this case, a user can select howto handle the lower-order bits cut away when compressed. Alternatively,it can be automatically selected based on their image conditions.

In step S122, the number of bits to be cut away determined whencompressed is read, and in step S1233, since they are not cut away, thenumber of bits to be cut away is set to 0.

In step S124, the compressed data is expanded and converted into the cutdata.

In step S125, the cut data is read, and in step S126, it is determinedwhether the cut bits are 0. If it is YES, the process proceeds to S1212without the superimposition process of lower-order bits. If it is NO,the process proceeds to step S127, and the entire bits are shifted tothe left side by the cut bits.

Then, in step S128, equation (1) of S103 is calculated and the averageb_(n) of adjacent pixel data a_(n−1) and a_(n+1) is calculated accordingt b_(n)=(a_(n−1)+a_(n+1))/2.

Then, in step S1210, the value (0,1,2,3) of c_(n) is selected accordingto the conditions of equations (2) through (5) of S103. Then, in stepS1211, the lower-order bits c_(n) is superimposed on the higher-orderbits a_(n) by adding them.

In step S1212, it is determined whether the process of all necessarypieces of pixel data is completed. If it is completed, the processproceeds to YES and terminates. If is not completed, the processproceeds to NO, the processes in S125 and after are repeated.

Although in this preferred embodiment, the bit width of pixel data is 12bits and is composed of 10 higher-order bits and 2 lower-order bits, thecomposition is not limited to this.

The lower-order bits can be generated using the correlation with pixeldata in the neighborhood of the target pixel data. Specifically, thelower-order bits can be generated using the correlation with adjacentpixel data on each side of the target pixel data. The lower-order bitscan also be generated using the correlation including pixel data otherthan one on each end of the target pixel data (vertically adjacent pixeldata of the target pixel data or adjacent pixel data adjacent to thetarget pixel data).

By adding such calculated lower-order bits, the edge of an image and thediscontinuity of gradation can be eliminated, further S/N can beimproved and a favorable gradation characteristic can be obtained.

The configuration can be modified as long as the edge of an image andthe discontinuity of gradation can be eliminated, further S/N can beimproved and a favorable gradation characteristic can be obtained.

Preferred Embodiment 3

In this preferred embodiment too, 12-bit pixel data composed ofhigher-order bits a (10 bits) and lower-order bits b (2 bits) areprocessed by a method using the correlation with adjacent colorcomponents.

FIG. 13 shows adjacent pixels in Bayer array. In order to determine thelower-order bits c_(n) Of a pixel a_(n) using the correlation withadjacent color components, the lower-order bits c_(n) is calculated anddetermined based on the correlation in pixel data between a pixelb_(n−1) adjacent to the pixel on the left side, a pixel b_(n−1−1) abovethe pixel on the right side and a pixel b_(n−1) above the pixel, whoselower-order bits are compensated.

A method for calculating the lower-order bits c_(n), based on thecorrelation with adjacent color components is described below withreference to FIG. 14.

In S141, the expanded 10-bit pixel data two bits of which are cut awaywhen compressed is shifted to the left side by two bits. The entire bitis shifted to the MSB side by two bits by shifting the bits in S141.Then, in S142, two lower-order bits b0 and b1 are set to 0. In thiscase, each of b0 and b1 is set to 0.

In S143, b_(n) is calculated according to equation (1) of S143. Firstly,b_(n)=(b_(n−1)+b_(n−1)−b_(n−1−1)) is calculated using the adjacent pixeldata b_(n−1), b_(n−1) and b_(n−1−1). Then, a value meeting theconditions is determined based on the result of b_(n) by referring toequations (2) through (5) of S143.

Equation (2) of S143 compares b_(n) with a preset threshold value 4(k₄). If b_(n)>k₄, c_(n) is set to 3 ([b0][b1]→[1][1]).

Equation (3) of S143 compares b_(n) with preset threshold values 4 (k₄)and 5(k₅). If k₅<b_(n)<k₄, c_(n) is set to 2 ([b0][b1]→[1][0]).

Equation (4) of S143 compares b_(n) with preset threshold values 5 (k₅)and 6(k₆). If k₆<b_(n)<k₅, c_(n) is set to 2 ([b0][b1]→[0][1]).

Equation (5) of S143 compares b_(n) with a preset threshold value 6(k₆). If b_(n)<k₆, c_(n) is set to 0 ([b0][b1]→[0][0]).

Then, in S144, the corrected lower-order bits c_(n), beingsuperimposition data determined in S143, is superimposed as thelower-order bits c_(n) of the pixel data.

In this case, each of the preset threshold values k₄, k₅ and k₆ is setin the range between the maximum value and the minimum value of b_(n). Auser can also set this is such a way to obtain a desirable image.

FIG. 15 shows the configuration for executing S141 through S154 shown inFIG. 14. An expansion unit 151 expands the 10-bit pixel data the twobits of which are cut away when compressed. Then, a bit shift unit 152performs the processes in S141 and S142. Data memory 153 stores theresult of S142. A coefficient (b_(n)) calculation unit 154 calls upb_(n−1), b_(n−1), and b_(n−1−1) stored in an adjacent pixel data memory155, and calculates b_(n) according to equation (1) of S143. Then, asuperimposition data calculation unit 156 determines c_(n) according tothe conditions of equations (2) through (5) of S143. After thecalculation, 12-bit pixel data can be generated by adding thesuperimposition data and the 10-bit left-shifted pixel data a_(n) storedin the data memory by the operation shown in S144.

Next, the flowchart shown in FIG. 16 is described below.

In step S161, it is determined whether lower-order bits are cut awaywhen compressing pixel data. In this case, information for determiningwhether lower-order bits are cut away is stored as part of compressioninformation, such as the number of bits cut away when compressed, andthe determination is made by referring to this information. If they arealready cut away, the process proceeds to S162. If they are not cut awayyet, the process proceeds to S163. In this case, a user can select howto handle the lower-order bits cut away when compressed. Alternatively,it can be automatically selected based on their image conditions.

In step S162, the number of bits to be cut away determined whencompressed is read, and in step S163, since they are not cut away, thenumber of bits to be cut away is set to 0.

In step S164, the compressed data is expanded and converted into the cutdata.

In step S165, the cut data is read, and in step S166, it is determinedwhether the cut bits are 0. If it is YES, the process proceeds to S1612without the superimposition process of lower-order bits. If it is NO,the process proceeds to step S167, and the entire bits are shifted tothe left side by the cut bits.

Then, in step S168, equation (1) of S143 is calculated, and acoefficient b_(n) is calculated according tb_(n)=(b_(n−1)+b_(n−1)−b_(n−1−1)) using of adjacent pixel data b_(n−1),B_(n−1) and B_(n−1−1).

Then, in step S169, the coefficient is compared with the threshold valueaccording to the conditions of equations (2) through (5) of S143 and instep S1610, the value (0,1,2,3) of C_(n) is selected. Then, in stepS1611, the lower-order bits c_(n) is superimposed on the higher-orderbits a_(n) by adding them.

In step S1612, it is determined whether the process of all necessarypieces of pixel data is completed. If it is completed, the processproceeds to YES and terminates. If is not completed, the processproceeds to NO, the processes in S165 and after are repeated.

Although in this preferred embodiment, the bit width of pixel data is 12bits and is composed of 10 higher-order bits and 2 lower-order bits, thecomposition is not limited to this.

The lower-order bits can be generated using the correlation with pixeldata in the neighborhood of the target pixel data. Specifically, thelower-order bits can be generated using the correlation with adjacentpixel data on each side of the target pixel data. The lower-order bitscan also be generated using the correlation including pixel data otherthan one on each end of the target pixel data (vertically adjacent pixeldata of the target pixel data or adjacent pixel data adjacent to thetarget pixel data).

By adding such calculated lower-order bits, the edge of an image and thediscontinuity of gradation can be eliminated, further S/N can beimproved and a favorable gradation characteristic can be obtained.

The configuration can be modified as long as the edge of an image andthe discontinuity of gradation can be eliminated, further S/N can beimproved and a favorable gradation characteristic can be obtained.

Preferred Embodiment 4

The image of an electronic camera device, such as a digital camera orthe like, is easily influenced by the luminance component of a G (green)signal. In a digital camera, a primary color Bayer array in whichG(green) is disposed checkerwise is popular, and the prediction accuracyof the G (green) signal in the R(red) and B(blue) positions leads to theimprovement of resolution/image quality.

When dividing an image signal into adjacent pixel blocks, each blockinformation can be indicated by a vector. It is known that the vectorinformation of R, G and B signal components in each block is highlycorrelated particularly in a low-saturation area. Conventionally, amethod for accurately predicting a G signal using a prediction valuebased on a G signal around R and B positions used for G correction andthe vector information of an R or B signal is known.

In this preferred embodiment, 12-bit pixel data composed of higher-orderbits a_(n) (10 bits) and lower-order bits c_(n) (2 bits) are processedby a method using the correlation with adjacent color components. Inorder to determine the lower-order bits c_(n) of a pixel an using thecorrelation with adjacent color components, the color componentcorrelation between two pieces of pixel data of adjacent pixels R(red)and B(blue) adjacent to G(green).

In this example, the following method is described in order to simplifythe process. FIG. 17 shows the primary color Bayer array, in which G(green) located to the side of R(red) and G (green) located to the sideof B(blue) is described as G_(r) and G_(b), respectively.

A method for calculating the lower-order bits, based on the correlationwith adjacent color components is described below with reference to FIG.18.

The lower-order bits of R(red) are determined based on the correlationwith adjacent G_(r), and the lower-order bits of G_(r) are determinedbased on the correlation with adjacent R(red). The lower-order bits of B(blue) are determined based on the correlation with adjacent Gb, and thelower-order bits are determined based on the correlation of adjacent B(blue).

For example, in the following description, the lower-order bits of anR(red) pixel are calculated based on the correlation with adjacent G_(r)pixel.

In S181, the expanded 10-bit pixel data of R(red), two bits of which arecut away when compressed, is shifted to the left side by two bits. Sincethe entire bit is shifted to the MSB side by two bits by shifting thebits in S181, in S182, each of two lower-order bits b0 and b1 are set to0.

In sl83, ω_(n) and the lower-order bits c_(n) are calculated accordingto equation 81) of S183. Firstly, ω_(n) ( )=A(G_(r)) is calculated usingthe respective higher-order bits (10-bit pixel data) of adjacent pixeldata R(red) and G_(r). In this case, A is a function to calculate acharacteristic based on the entire image shot by a digital camera andcalculate the correlation among R(red), G(green) and B(blue). As thefunction to calculate the correlation, a table can be prepared.Alternatively, a linear function, an n-degree function, an n-degreepolynomial, regression analysis or the like can be calculated and used.A can also be a function based on the correlation with R(red) adjacentto G_(r) or B(blue) adjacent to G_(b), in the primary color Bayer array.

Then, c_(n) is determined by assigning pixel data adjacent to the targetpixel to the function ω_(n)( ). In this case too, c_(n) is obtained bypreparing a table or the like. For example, if two lower-order bits aredetermined, as in preferred embodiment 2, a threshold can be determinedand one is selected from 0 through 3.

Then, the higher-order bits a_(n−1) of the target adjacent pixel iscalculated using ω_(n) and the lower-order bits c_(n) is calculated.Then, in S184, the corrected lower-order bits c_(n), beingsuperimposition data determined in S183 is superimposed on thehigher-order bits a_(n).

FIG. 19 shows the configuration for executing S181 through S184 shown inFIG. 18. An expansion unit 191 expands the 10-bit pixel data, two bitsof which is cut away. Then, a bit shift unit 192 performs the processesin S181 and S182. Data memory 193 stores the result of S182. Asuperimposition coefficient reading unit 194 calls up a_(n−1) anda_(n+1) among all the pieces of pixel data stored in superimpositioncoefficient memory 195, and calculates c_(n) according to S183. Then, asuperimposition data calculation unit 196 determines c_(n). After thecalculation, the superimposition data is added to the 10-bitleft-shifted pixel data a_(n) stored in the data memory by the operationshown in S184 to generate 12-bit pixel data.

Next, the flowchart shown in FIG. 20 is described below.

In step S201, it is determined whether lower-order bits are cut awaywhen compressing pixel data. In this case, information for determiningwhether lower-order bits are cut away is stored as part of compressioninformation, such as the number of bits cut away when compressed, andthe determination is made by referring to this information. If they arealready cut away, the process proceeds to S202. If they are not cut awayyet, the process proceeds to S203. In this case, a user can select howto handle the lower-order bits cut away when compressed. Alternatively,it can be automatically selected based on their image conditions.

In step S202, the number of bits to be cut away determined whencompressed is read, and in step S203, since they are not cut away, thenumber of bits to be cut away is set to 0.

In step S204, the compressed data is expanded and converted into the cutdata.

In step S205, the cut data is read, and in step S206, it is determinedwhether the cut bits are 0. If it is YES, the process proceeds to S2011without the superimposition process of lower-order bits. If it is NO,the process proceeds to step S207, and the entire bits are shifted tothe left side by the cut bits. Then, in step S208, the superimpositiondata is read, and in step S209, the lower-order bits c_(n) is calculatedby the operation in S203. Then, in step S2010, the lower-order bitsc_(n) is superimposed on the higher-order bits an by adding them.

In step S2011, it is determined whether the process of all necessarypieces of pixel data is completed. If it is completed, the processproceeds to YES and terminates. If is not completed, the processproceeds to NO, the processes in S205 and after are repeated.

Although in this preferred embodiment, the bit width of pixel data is 12bits and is composed of 10 higher-order bits and 2 lower-order bits, thecomposition is not limited to this. According to the above-mentionedconfiguration, the lower-order bits can be generated using thecorrelation with pixel data with a color component different from thatof the target pixel data in the primary color Bayer array.

The lower-order bits can be generated by calculating color componentcorrelation of the entire pixel data and using the color componentcorrelation. The lower-order bits can also be generated by dividingpixel data, calculating the color component correlation of the dividedintra-block image data and using the correlation. By retrieving datafrom a table using the value calculated based on the color componentcorrelation, detecting the same number of lower-order bits as those cutaway and adding the lower-order bits, the edge of an image and thediscontinuity of gradation can be eliminated and S/N can be improved.

(Variation)

By superimposing data on data with a small number of gradations, thenumber of gradation can be increased. As this effect, an unnaturalgradation change caused by an image processing can be eliminated.

For example, the tone jump at the edge and in the gradation part can bemitigated.

The present invention can also be implemented by generating a controlprogram for enabling the CPU of the standard computer described earlierto execute the processes in the flowcharts shown in FIGS. 2, 3, 8, 12,16 and 20, storing it in a computer-readable storage medium and enablingthe computer to read the program from the storage medium and the CPU toexecute it.

FIG. 21 shows examples of storage media on which is recorded such acontrol program which can be read by a computer system. As such astorage medium, a storage device 2102, such as ROM, a hard disk device,etc., built in or externally provided for a computer system 2101 as anauxiliary device, a portable storage medium 2104, such as a flexibledisk, a magneto-optical disk (MO), CD-ROM, DVD-ROM, etc., the recordedcontrol program of which can be read by inserting it a medium drivingdevice 2103 provided for the computer system 2102 or the like can beused.

It can also be a storage device 2107 provided for the computer system,which is connected to the computer system 2101 via a communication line2105 and functions as a program server 2106. In this case, atransmission signal obtained by modulating a data signal representingthe control program by a carrier wave is transmitted to the computersystem 2101 via the communication line 2105, being a transmissionmedium, from the program server 2106. The computer system 2101 canexecute this control program by the CPU of the computer system 2101 bydemodulating the received transmission signal and reproducing thecontrol program.

Although in the above-mentioned preferred embodiments, a digital camerais used, the present invention is applicable to any other imageprocessing device as long as the subject matter of the present inventionset forth in What is claimed is not deviated.

The present invention provided an image processing device foreliminating the edge of an image and the discontinuity of gradation byexpanding compressed image data whose lower-order bits are uniformly cutaway as camera noise components when storing, calculating the data oflower-order bits for correcting the expanded pixel data afterbit-shifting the expanded data and replacing the cut components(lower-order bits) with it, further improving S/N to obtain a favorablegradation characteristic.

As described above, the number of gradation can be increased by addingSuperimposing) the corrected lower-order bits to (on) the image datawhose lower-order bits are cut away. Thus, in an image to which imageprocessing, such as compression/expansion or the like, the edge of animage and the discontinuity of gradation can be eliminated, S/N isimproved and a favorable gradation characteristic can be obtained,thereby preventing an unnatural level change (tone jump) from occurring.

1. An image processing device for predicting lower-order bits of targetpixel data, based on one or more pieces of pixel data constituting imagedata, comprising: a lower-order bit calculation unit for calculating thelower-order bits of the target pixel data, based on the one or morepieces of pixel data constituting the image data and specifying thecalculated lower-order bits of the target pixel data as correctedlower-order bits; and a lower-order bit superimposition unit for addingthe corrected lower-order bits calculated by the lower-order bitcalculation unit to higher-order bits of the target pixel data.
 2. Theimage processing device according to claim 1, wherein said lower-orderbit calculation unit calculates the corrected lower-order bits, based onhigher-order bits of the pixel data.
 3. The image processing deviceaccording to claim 1, wherein said lower-order bit calculation unitdetermines a weighting coefficient of a value of the pixel data andcalculates the corrected lower-order bits by multiplying thehigher-order bits of the target pixel data by the weighting coefficient.4. The image processing device according to claim 1, wherein saidlower-order bit calculation unit calculates correlation with a pixelvalue of pixel data in the neighborhood of the target pixel data andcalculates the corrected lower-order bits, based on the correlation. 5.The image processing device according to claim 4, wherein saidlower-order bit calculation unit calculates the correlation bycalculating an average of higher-order bits of adjacent pixels on eachside of the target pixel.
 6. The image processing device according toclaim 4, wherein said lower-order bit calculation unit calculates thecorrelation by calculating an average of higher-order bits of pixels inthe neighborhood of the target pixel.
 7. The image processing deviceaccording to claim 1, wherein said lower-order bit calculation unitcalculates a first color component correlation, based on thehigher-order bits of a pixel with a different color component in primarycolor Bayer array in the neighborhood of the target pixel, furthercalculates a second color component correlation, based on the result ofthe first color component correlation and the higher-order bits of thepixel and calculates the corrected lower-order bits, based on the secondcolor component correlation.
 8. The image processing device according toclaim 7, wherein said lower-order bit calculation unit calculates thefirst color component correlation by calculating a function, based oncorrelation using the higher-order bits of pixels of the entire imagedata.
 9. The image processing device according to claim 7, wherein saidlower-order bit calculation unit calculates the first color componentcorrelation by dividing the image data and calculating a function foreach divided block, based on correlation with the higher-order bits of apixel in the divided block.
 10. The image processing device according toclaim 7, wherein said lower-order bit calculation unit calculates thesecond color component correlation by calculating the correctedlower-order bits using a table prepared beforehand using valuescalculated based on the first color component correlation.
 11. Acomputer-readable storage medium storing a program for enabling acomputer to execute image processing of predicting lower-order bits oftarget pixel data, based on one or more pieces of pixel dataconstituting image data, the image processing comprising: calculatingthe lower-order bits of the target pixel data, based on the one or morepieces of pixel data constituting the image data and specifying thecalculated lower-order bits of the target pixel data as correctedlower-order bits; and superimposing the corrected lower-order bits onhigher-order bits of the target pixel data.
 12. The storage mediumaccording to claim 11, wherein the program enables the computer tocalculate the corrected lower-order bits, based on higher-order bits ofthe pixel data.
 13. A computer data signal embodied in a carrier wave,for representing a program for enabling a computer to execute imageprocessing of predicting lower-order bits of target pixel data, based onone or more pieces of pixel data constituting image data, the imageprocessing comprising: calculating the lower-order bits of the targetpixel data, based on the one or more pieces of pixel data constitutingthe image data and specifying the calculated lower-order bits of thetarget pixel data as corrected lower-order bits; and superimposing thecorrected lower-order bits on higher-order bits of the target pixeldata.
 14. The computer data signal according to claim 13, wherein theprogram enables the computer to calculate the corrected lower-orderbits, based on higher-order bits of the pixel data.